Editing faces in videos

Editing faces in movies is of interest in the special effects industry. We aim at producing effects such as the addition of accessories interacting correctly with the face or replacing the face of a stuntman with the face of the main actor. The system introduced in this thesis is based on a 3D generative face model. Using a 3D model makes it possible to edit the face in the semantic space of pose, expression, and identity instead of pixel space, and due to its 3D nature allows a modelling of the light interaction. In our system we first reconstruct the 3D face, which is deforming because of expressions and speech, the lighting, and the camera in all frames of a monocular input video. The face is then edited by substituting expressions or identities with those of another video sequence or by adding virtual objects into the scene. The manipulated 3D scene is rendered back into the original video, correctly simulating the interaction of the light with the deformed face and virtual objects. We describe all steps necessary to build and apply the system. This includes registration of training faces to learn a generative face model, semi-automatic annotation of the input video, fitting of the face model to the input video, editing of the fit, and rendering of the resulting scene. While describing the application we introduce a host of new methods, each of which is of interest on its own. We start with a new method to register 3D face scans to use as training data for the face model. For video preprocessing a new interest point tracking and 2D Active Appearance Model fitting technique is proposed. For robust fitting we introduce background modelling, model-based stereo techniques, and a more accurate light model

Novel Interactions between Actin and the Proteasome Revealed by Complex Haploinsufficiency

Saccharomyces cerevisiae has been a powerful model for uncovering the landscape of binary gene interactions through whole-genome screening. Complex heterozygous interactions are potentially important to human genetic disease as loss-of-function alleles are common in human genomes. We have been using complex haploinsufficiency (CHI) screening with the actin gene to identify genes related to actin function and as a model to determine the prevalence of CHI interactions in eukaryotic genomes. Previous CHI screening between actin and null alleles for non-essential genes uncovered ∼240 deleterious CHI interactions. In this report, we have extended CHI screening to null alleles for essential genes by mating a query strain to sporulations of heterozygous knock-out strains. Using an act1Δ query, knock-outs of 60 essential genes were found to be CHI with actin. Enriched in this collection were functional categories found in the previous screen against non-essential genes, including genes involved in cytoskeleton function and chaperone complexes that fold actin and tubulin. Novel to this screen was the identification of genes for components of the TFIID transcription complex and for the proteasome. We investigated a potential role for the proteasome in regulating the actin cytoskeleton and found that the proteasome physically associates with actin filaments in vitro and that some conditional mutations in proteasome genes have gross defects in actin organization. Whole-genome screening with actin as a query has confirmed that CHI interactions are important phenotypic drivers. Furthermore, CHI screening is another genetic tool to uncover novel functional connections. Here we report a previously unappreciated role for the proteasome in affecting actin organization and function

Functional Expression of Human Adenine Nucleotide Translocase 4 in Saccharomyces Cerevisiae

The adenine nucleotide translocase (ANT) mediates the exchange of ADP and ATP across the inner mitochondrial membrane. The human genome encodes multiple ANT isoforms that are expressed in a tissue-specific manner. Recently a novel germ cell-specific member of the ANT family, ANT4 (SLC25A31) was identified. Although it is known that targeted depletion of ANT4 in mice resulted in male infertility, the functional biochemical differences between ANT4 and other somatic ANT isoforms remain undetermined. To gain insight into ANT4, we expressed human ANT4 (hANT4) in yeast mitochondria. Unlike the somatic ANT proteins, expression of hANT4 failed to complement an AAC-deficient yeast strain for growth on media requiring mitochondrial respiration. Moreover, overexpression of hANT4 from a multi-copy plasmid interfered with optimal yeast growth. However, mutation of specific amino acids of hANT4 improved yeast mitochondrial expression and supported growth of the AAC-deficient yeast on non-fermentable carbon sources. The mutations affected amino acids predicted to interact with phospholipids, suggesting the importance of lipid interactions for function of this protein. Each mutant hANT4 and the somatic hANTs exhibited similar ADP/ATP exchange kinetics. These data define common and distinct biochemical characteristics of ANT4 in comparison to ANT1, 2 and 3 providing a basis for study of its unique adaptation to germ cells

In silico toxicology protocols

The present publication surveys several applications of in silico (i.e., computational) toxicology approaches across different industries and institutions. It highlights the need to develop standardized protocols when conducting toxicity-related predictions. This contribution articulates the information needed for protocols to support in silico predictions for major toxicological endpoints of concern (e.g., genetic toxicity, carcinogenicity, acute toxicity, reproductive toxicity, developmental toxicity) across several industries and regulatory bodies. Such novel in silico toxicology (IST) protocols, when fully developed and implemented, will ensure in silico toxicological assessments are performed and evaluated in a consistent, reproducible, and well-documented manner across industries and regulatory bodies to support wider uptake and acceptance of the approaches. The development of IST protocols is an initiative developed through a collaboration among an international consortium to reflect the state-of-the-art in in silico toxicology for hazard identification and characterization. A general outline for describing the development of such protocols is included and it is based on in silico predictions and/or available experimental data for a defined series of relevant toxicological effects or mechanisms. The publication presents a novel approach for determining the reliability of in silico predictions alongside experimental data. In addition, we discuss how to determine the level of confidence in the assessment based on the relevance and reliability of the information

GraphTrack: fast and globally optimal tracking in videos

In video post-production it is often necessary to track in-terest points in the video. This is called off-line tracking, be-cause the complete video is available to the algorithm and can be contrasted with on-line tracking, where an incom-ing stream is tracked in real time. Off-line tracking should be accurate and – if used interactively – needs to be fast, preferably faster than real-time. We describe a 50 to 100 frames per second off-line tracking algorithm, which glob-ally maximizes the probability of the track given the com-plete video. The algorithm is more reliable than previous methods because it explains the complete frames, not only the patches of the final track, making as much use of the data as possible. It achieves efficiency by using a greedy search strategy with deferred cost evaluation, focusing the compu-tational effort on the most promising track candidates while finding the globally optimal track. 1

Weight, Sex, and Facial Expressions : on the Manipulation of Attributes in Generative 3D Face Models

Generative 3D Face Models are expressive models with applicationsin modelling and editing. They are learned from example faces, and offer a compactrepresentation of the continuous space of faces. While they have proven tobe useful as strong priors in face reconstruction they remain to be difficult to usein artistic editing tasks. We describe a way to navigate face space by changingmeaningful parameters learned from the training data. This makes it possible tofix attributes such as height, weight, age, expression or ‘lack of sleep’ while lettingthe infinity of unfixed other attributes vary in a statistically meaningful way.We propose an inverse approach based on learning the distribution of faces inattribute space. Given a set of target attributes we then find the face which has thetarget attributes with high probability, and is as similar as possible to the inputface

On Compositional Image Alignment, with an Application to Active Appearance Models

Efficient and accurate fitting of Active AppearanceModels (AAM) is a key requirement for many applications.The most efficient fitting algorithm today is Inverse CompositionalImage Alignment (ICIA). While ICIA is extremelyfast, it is also known to have a small convergence radius.Convergence is especially bad when training and testingimages differ strongly, as in multi-person AAMs. We describe“forward” compositional image alignment in a consistentframework which also incorporates methods previouslytermed “inverse” compositional, and use it to developtwo novel fitting methods. The first method, CompositionalGradient Descent (CoDe), is approximately fourtimes slower than ICIA, while having a convergence radiuswhich is even larger than that achievable by direct Quasi-Newton descent. An intermediate convergence range withthe same speed as ICIA is achieved by LinCoDe, the secondnew method. The success rate of the novel methods is 10 to20 times higher than that of the original ICIA method

Old Yellow Enzyme Protects the Actin Cytoskeleton from Oxidative Stress

Old Yellow Enzyme (OYE) has long served as a paradigm for the study of flavin-containing NADPH oxido-reductases and yet its physiological role has remained a mystery. A two-hybrid interaction between Oye2p and actin led us to investigate a possible function in the actin cytoskeleton. We found that oye deletion strains have an overly elaborate actin cytoskeleton that cannot be attributed to changes in actin concentration but likely reflect stabilization of actin filaments, resulting in excessive actin assembly. Cells expressing the actin mutant act1-123p, which has a weakened interaction with Oye2p, show comparable defects in actin organization to the oye deletion strain that can be suppressed by overexpression of Oye2p. Similarly, mutation of either conserved cysteine of the potential disulfide pair Cys285-Cys374 in actin completely suppresses the actin organization defect of the oyeΔ phenotype. Strains lacking Oye function are also sensitive to oxidative stress as induced by H(2)O(2), menadione, and diamide treatment. Mutation of either Cys285 or Cys374 of actin suppresses the sensitivity of oyeΔ strains to oxidative stress and in fact confers super-resistance to oxidative stress in otherwise wild-type strains. These results suggest that oxidative damage to actin, like that which has been observed in irreversibly sickled red blood cells, may be a general phenomenon and that OYE functions to control the redox state of actin thereby maintaining the proper plasticity of the actin cytoskeleton. In addition to uncovering a long sought biological function for Old Yellow Enzyme, these results establish that cellular sensitivity to oxidative stress can in part be directly attributed to a specific form (C285-C374 disulfide bond formation) of oxidative damage to actin